Analyzing Transformer Dynamics as Movement through Embedding Space

Transformer language models exhibit intelligent behaviors such as understanding natural language, recognizing patterns, acquiring knowledge, reasoning, planning, reflecting and using tools. This paper explores how their underlying mechanics give rise to intelligent behaviors. We adopt a systems approach to analyze Transformers in detail and develop a mathematical framework that frames their dynamics as movement through embedding space. This novel perspective provides a principled way of thinking about the problem and reveals important insights related to the emergence of intelligence: 1. At its core the Transformer is a Embedding Space walker, mapping intelligent behavior to trajectories in this vector space. 2. At each step of the walk, it composes context into a single composite vector whose location in Embedding Space defines the next step. 3. No learning actually occurs during decoding; in-context learning and generalization are simply the result of different contexts composing into different vectors. 4. Ultimately the knowledge, intelligence and skills exhibited by the model are embodied in the organization of vectors in Embedding Space rather than in specific neurons or layers. These abilities are properties of this organization. 5. Attention's contribution boils down to the association-bias it lends to vector composition and which influences the aforementioned organization. However, more investigation is needed to ascertain its significance. 6. The entire model is composed from two principal operations: data independent filtering and data dependent aggregation. This generalization unifies Transformers with other sequence models and across modalities. Building upon this foundation we formalize and test a semantic space theory which posits that embedding vectors represent semantic concepts and find some evidence of its validity

Full Page Handwriting Recognition via Image to Sequence Extraction

We present a Neural Network based Handwritten Text Recognition (HTR) model architecture that can be trained to recognize full pages of handwritten or printed text without image segmentation. Being based on an Image to Sequence architecture, it can be trained to extract text present in an image and sequence it correctly without imposing any constraints on language, shape of characters or orientation and layout of text and non-text. The model can also be trained to generate auxiliary markup related to formatting, layout and content. We use character level token vocabulary, thereby supporting proper nouns and terminology of any subject. The model achieves a new state-of-art in full page recognition on the IAM dataset and when evaluated on scans of real world handwritten free form test answers - a dataset beset with curved and slanted lines, drawings, tables, math, chemistry and other symbols - it performs better than all commercially available HTR APIs. It is deployed in production as part of a commercial web application

POWERING CUSTOMER CONNECTIVITY OF PRODUCTS WITH LAST MILE SUPPORT SERVICES: EVIDENCES FROM INDIAN PASSENGER CAR MANUFACTURING

The paper highlights post-production customer connectivity processes in an automobile manufacturing set-up. Identity of post-production processes has been established as a unique in company service operations or pseudo-manufacturing activities. Consequently this research observes that handling these activities purely with a manufacturing focus or with a pure service focus would not be that effective. This paper in-turn proposes a new approach to powering the customer connectivity process in the last mile of a product’s journey from manufacturing to market. The scenario has been addressed in three domains, viz., Post-Production pre-market functions, Product Installation, and Product Liabilities. Through a composite process map of the post-production situations it captures the order winning elements  to pick up opportunities for the possible reinforcement of adding value for money in the last mile activities on the product from company to customer.

Bilateral post traumatic thalamic infarct: Rare case report

Bilateral thalamic infarction is rarely seen in cases with artery of Percheron embolization. Artery of Percheron is a rare variant of arterial supply to both thalamus and midbrain. Occlusion of this artery following trauma and causing bilateral thalamic infarction is rare and very few cases are reported in world literature

A Clinical Practice Guideline for the Management of Patients With Degenerative Cervical Myelopathy: Recommendations for Patients With Mild, Moderate, and Severe Disease and Nonmyelopathic Patients With Evidence of Cord Compression.

Study Design: Guideline development. Objectives: The objective of this study is to develop guidelines that outline how to best manage (1) patients with mild, moderate, and severe myelopathy and (2) nonmyelopathic patients with evidence of cord compression with or without clinical symptoms of radiculopathy. Methods: Five systematic reviews of the literature were conducted to synthesize evidence on disease natural history; risk factors of disease progression; the efficacy, effectiveness, and safety of nonoperative and surgical management; the impact of preoperative duration of symptoms and myelopathy severity on treatment outcomes; and the frequency, timing, and predictors of symptom development. A multidisciplinary guideline development group used this information, and their clinical expertise, to develop recommendations for the management of degenerative cervical myelopathy (DCM). Results: Our recommendations were as follows: (1) "We recommend surgical intervention for patients with moderate and severe DCM." (2) "We suggest offering surgical intervention or a supervised trial of structured rehabilitation for patients with mild DCM. If initial nonoperative management is pursued, we recommend operative intervention if there is neurological deterioration and suggest operative intervention if the patient fails to improve." (3) "We suggest not offering prophylactic surgery for non-myelopathic patients with evidence of cervical cord compression without signs or symptoms of radiculopathy. We suggest that these patients be counseled as to potential risks of progression, educated about relevant signs and symptoms of myelopathy, and be followed clinically." (4) "Non-myelopathic patients with cord compression and clinical evidence of radiculopathy with or without electrophysiological confirmation are at a higher risk of developing myelopathy and should be counselled about this risk. We suggest offering either surgical intervention or nonoperative treatment consisting of close serial follow-up or a supervised trial of structured rehabilitation. In the event of myelopathic development, the patient should be managed according to the recommendations above." Conclusions: These guidelines will promote standardization of care for patients with DCM, decrease the heterogeneity of management strategies and encourage clinicians to make evidence-informed decisions

The antiinflammatory potential of phenolic compounds from Emblica officinalis L. in rat

Antiinflammatory effects of phenolic compounds from Emblica officinalis were evaluated in carrageenan and cotton pellet induced acute and chronic inflammatory animal model. Fractions of E. officinalis containing free (FPEO) and bounded (BPEO) phenolic compounds were assessed by HPLC technique. The free and bound phenolic compounds were studied for their acute and chronic antiinflammatory activity at dose level of 20 and 40 mg/kg. The carrageenan induced acute inflammation was assessed by measuring rat paw volume at different time of intervals. Further, cotton pellet induced chronic inflammation was assessed by granulomatous tissue mass estimation along with the estimation of tissue biomarker changes (i.e. lipid peroxidation, reduced glutathione, myeloperoxidase and plasma extravasation). The results indicated that in both acute and chronic inflammation, FPEO and BPEO show reduction in the inflammation, but significant effects was observed only at high doses of both fractions which was comparable to diclofenac treated group. In conclusion, phenolic compounds of E. officinalis may serve as potential herbal candidate for amelioration of acute and chronic inflammation due to their modulatory action of free radicals

Decline in subarachnoid haemorrhage volumes associated with the first wave of the COVID-19 pandemic

BACKGROUND: During the COVID-19 pandemic, decreased volumes of stroke admissions and mechanical thrombectomy were reported. The study\u27s objective was to examine whether subarachnoid haemorrhage (SAH) hospitalisations and ruptured aneurysm coiling interventions demonstrated similar declines. METHODS: We conducted a cross-sectional, retrospective, observational study across 6 continents, 37 countries and 140 comprehensive stroke centres. Patients with the diagnosis of SAH, aneurysmal SAH, ruptured aneurysm coiling interventions and COVID-19 were identified by prospective aneurysm databases or by International Classification of Diseases, 10th Revision, codes. The 3-month cumulative volume, monthly volumes for SAH hospitalisations and ruptured aneurysm coiling procedures were compared for the period before (1 year and immediately before) and during the pandemic, defined as 1 March-31 May 2020. The prior 1-year control period (1 March-31 May 2019) was obtained to account for seasonal variation. FINDINGS: There was a significant decline in SAH hospitalisations, with 2044 admissions in the 3 months immediately before and 1585 admissions during the pandemic, representing a relative decline of 22.5% (95% CI -24.3% to -20.7%, p\u3c0.0001). Embolisation of ruptured aneurysms declined with 1170-1035 procedures, respectively, representing an 11.5% (95%CI -13.5% to -9.8%, p=0.002) relative drop. Subgroup analysis was noted for aneurysmal SAH hospitalisation decline from 834 to 626 hospitalisations, a 24.9% relative decline (95% CI -28.0% to -22.1%, p\u3c0.0001). A relative increase in ruptured aneurysm coiling was noted in low coiling volume hospitals of 41.1% (95% CI 32.3% to 50.6%, p=0.008) despite a decrease in SAH admissions in this tertile. INTERPRETATION: There was a relative decrease in the volume of SAH hospitalisations, aneurysmal SAH hospitalisations and ruptured aneurysm embolisations during the COVID-19 pandemic. These findings in SAH are consistent with a decrease in other emergencies, such as stroke and myocardial infarction